Video-related meta data engine, system and method

ABSTRACT

A video engine, system and method for use with a video player, including a presenter that presents at least two audio/visual works, at least one software application capable of at least one metadata-related interaction with the audio/visual works, communication points over which the audio/visual works are received, and over which at least a portion of the at least one meta-data interaction occurs, and a hierarchical taxonomy that effects a common metadata reference to each recurrence of a particular object across the audio/visual works, and across each of the at least one metadata-related interaction. The video engine, system, and method may additionally include a prioritization data for use with the metadata.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.11/981,838, entitled “Video-Related Meta Data Engine, System andMethod,” filed Oct. 30, 2007; the entire contents of which isincorporated by reference herein as if each set forth in its entirety.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention is directed to video and metadata and, moreparticularly, to a video-related metadata engine, system and method.

The present invention is related to concurrently filed U.S. patentapplication Ser. No. 11/981,839, entitled “SYSTEM AND METHOD FORCREATION AND MANAGEMENT OF ADVERTISING INVENTORY USING METADATA” filedOct. 31, 2007, the entire contents of which are incorporated herein byreference. The present invention is related to concurrently filed U.S.patent application Ser. No. 11/981,763, entitled “OPEN API VIDEO SYSTEMAND METHOD OF MAKING AND USING SAME” filed Oct. 31, 2007, the entirecontents of which are incorporated herein by reference.

DESCRIPTION OF THE BACKGROUND

Present endeavors to create applications for operation on metadataassociated with audio/visual works suffers from the drawback that onlycertain audio/visual works can have each application applied thereto, inpart because there must be an agreement between the application providerand the metadata provider as to the terminology used in the metadata toallow for operation by the application.

Thus, there exists a need for a video engine interoperable with a commonnomenclature across all applications and audio/visual works, therebyallowing for standardized interaction between any application and anyaudio/visual work.

SUMMARY OF THE INVENTION

The present invention includes at least a video engine, system andmethod for use with a video player, including a presenter that presentsat least two audio/visual works, at least one software applicationcapable of at least one metadata-related interaction with theaudio/visual works, communication points over which the audio/visualworks are received, and over which at least a portion of the at leastone meta-data interaction occurs, and a hierarchical taxonomy thateffects a common metadata reference to each recurrence of a particularobject across the audio/visual works, and across each of the at leastone metadata-related interaction. The video engine, system, and methodmay additionally include a prioritization data for use with themetadata.

Thus, the present invention provides a video engine interoperable with acommon nomenclature across all applications and audio/visual works,thereby allowing for standardized interaction between any applicationand any audio/visual work.

BRIEF DESCRIPTION OF THE FIGURES

The present invention will be described hereinbelow in conjunction withthe following figures, in which like numerals represent like items, andwherein:

FIG. 1 illustrates a video player in accordance with the presentinvention; and

FIG. 2 illustrates a video player-related hierarchical taxonomy inaccordance with the present invention.

DETAILED DESCRIPTION OF THE INVENTION

It is to be understood that the figures and descriptions of the presentinvention have been simplified to illustrate elements that are relevantfor a clear understanding of the present invention, while eliminating,for the purposes of clarity, many other elements found in typicalinteractive, metadata, and video play systems and methods. Those ofordinary skill in the art will recognize that other elements aredesirable and/or required in order to implement the present invention.However, because such elements are well known in the art, and becausethey do not facilitate a better understanding of the present invention,a discussion of such elements is not provided herein.

FIG. 1 is a block diagram illustrating an audio/video media player 10(hereinafter videoplayer) having associated therewith software 12 andhardware 14, in the form of a video engine 16, for the playing ofaudio/visual works 18 on the videoplayer 10 (the software and hardwarereferred to hereinafter as a video engine). The videoplayer as discussedherein may include any type of videoplayer that makes use of anyaudio/video media for playing on the videoplayer. The videoplayer maybe, but is not limited to, televisions, desktop and laptop computers,personal digital assistants (PDA), personal entertainment devices, suchas IPODs, mobile telephones, and the like, typically having videoprocessing and play capabilities.

The video engine 16 of the present invention operates in conjunctionwith a video player 10 in order to present audio/visual works to a userof the videoplayer. The video engine includes all hardware necessary toexecute the playing of the video on the videoplayer, and additionallyincludes one or more software applications capable of presenting notonly audio/visual works, but additionally capable of manipulating userinteraction with such audio/visual works, or manipulating suchaudio/visual works themselves. The software aspects and applications ofthe video engine may be partly or entirely remote from the videoplayer,such as to allow for development of applications, interactions or datamanipulations remote from the audio/visual work(s), or may be entirelylocal to the videoplayer. Such software applications may interact withthe audio/visual works via, for example, locally or remotely generatedmeta data 24 embedded in or associated with the audio/visual work, or ameta data stream received by the video player and video engineseparately from, and or in conjunction with, the audio/visual work oraudio/visual work stream.

The video engine, as used herein, may include any software applicationcapable of receiving audio/visual works and instructions associatedtherewith, and additionally capable of relaying instructions and/orcommands to an from at least one manipulation of or interaction withsuch audio/visual work. A video engine may be, but is not limited to, adigital video recorder, a computer hard drive in association with one ormore processors, a micro processor in conjunction with a videoprocessor, or the like. The video engine may include typical hardwareand/or software to allow for viewing of or interaction with anaudio/visual work, such as a hard drive, random access memory, flashmemory, or the like, and may receive and/or communicate the audio/visualwork, commands, interactions and/or instructions from and tocommunication points via, for example, satellite communication, radiocommunication, wired communication, infrared communication, coaxialcable communication, WIFI communication, WiMAX communication, LANcommunication, WAN communication, telephonic communication, DSLcommunication, Ethernet communication, or the like as would be known tothose skilled in the art.

Metadata 24 as used herein, encompasses any type of computer readableinformation that may be associated with an audio/visual work, any objecttherein, or any portion thereof, including the formation or portionsthereof, or that may be used for interaction thereupon or therewith, aswill be understood by one of ordinary skill in the art. Metadata, asused herein, is defined to include any type of executable code, computerlanguage code (such as xml or html, object code and/or source code), or“mash-up” data (program associated data integral with data of interest)that may be associated with or into an audio/visual work. Metadatafurther includes metadata created through the use of the presentinvention and pre-existing metadata that may be, form part of, or beassociated with the audio-visual works on which the present invention isoperated. As discussed hereinthroughout, interactivity with anaudio/visual work may include manipulation of the audio/visual workitself, manipulation of a menu, overlay, or the like associated withsuch audio/visual work, off-line accessing or content requests forcontent associated with such audio/visual work or with such interactionwith such audio/visual work, and peer to peer interactivity, forexample. As such, the present invention makes available interactivitywith an audio/visual work of any type known to one of ordinary skill inthe art, and interactivity between any entities as such interaction mayrelate to an audio/visual work, including server entities known to thoseskilled in the art, and the obtaining of information related to, basedin, or related to information related to or based in, an audio/visualwork, over any communications media, as will be apparent to one ofordinary skill in the art.

As shown in FIG. 2, the present invention thus is and includes withinthe video engine 16 a hierarchical taxonomy 50 for making commonreference to items, objects, or portions 52 of and within audio/visualworks across multiple audio/visual works 18 and across multipleinteractivity planes 54 and/or interactive applications 12 forinteracting with ones of the multiple audio/visual works. Thus, in anexemplary embodiment of the present invention, one, several, or everyobject in every audiovisual work, and/or one or more portions or topicsof portions of one or more audio/visual works, may be assigned a commonnomenclature having set nomenclature to reference such object(s) at eachof different levels of a hierarchy, with such nomenclature, andnomenclature at such levels, being common across all audio/visual worksin accordance with the metadata indicative of each object or portion, asreferred to at each such hierarchical level, in each such audio/visualwork.

Thus, in the present invention the metadata associated with each suchaudio/visual work is built, such as by manual entry after review of theaudio/visual work, or by automated object (audio and/or video and/orcharacter) recognition, which may employ a “crawler” to review manyaudio/visual works, to use the proper, common nomenclature for eachobject or portion within and of the work upon every reference to thatobject or portion across all audio/visual works making reference to thatobject or portion. For example, the hierarchy may include any number oftop level categories, such as “fashion”, “automotive”, “health andleisure”, and the like. Needless to say, the present invention is by nomeans limited to the aforementioned list, and in fact includes top levelhierarchal nomenclature in each category of an object or video portionthat may appear in an audio/visual work, as will be apparent to thoseskilled in the art. The hierarchical nomenclature of the presentinvention is systematically applied with a particular view to thoseitems in an audio/video work that might be of most interest, such as toa consumer or advertiser, as will also be apparent to one or ordinaryskill in the art.

In the example mentioned above, the top level hierarchy “fashion” mayinclude any number of subcategories, such as “suits”, “dresses”,“shoes”, “accessories”, and the like. Again, it goes without saying thatthe aforementioned list is in no way limiting, but rather the importanceof the aforementioned list lies in the fact that the nomenclature ineach of the hierarchal categories within the hierarchical taxonomy ofthe video engine is not varied, with respect to its metadata applicationto objects and portions within audio/visual works, nor is it variedacross audio/visual works.

Continuing with the example hereinabove, the sub category “accessories”may include, for example, “purses”. Thus, the metadata associated withany audio/video work having an object therein qualifying as a pursewould either be manually recognized or automatically recognized, andlabeled for use via the video engine in accordance with the nomenclaturehierarchy as 1. Fashion, 1A. Accessories, 1AA. Purses.

Upon reaching any particular level of the nomenclature hierarchy, morespecific references may or may not be employed based on the intended useof the audio/visual work, but any more specific references must beemployed in accordance with the nomenclature hierarchy. For example, aconsumer who watches audio/visual words may wish to employ thenomenclature to access video snippets based on references to thatconsumer's favorite baseball team. In such a case, the initial hierarchyemployed by the application that accesses, using associated metadata,particular vide snippets may be “professional sports”, the sub-categorymay be “major league baseball”, and a specific reference may be“Philadelphia Phillies”. No additionally specific references may benecessary for such a use, and thus may not be provided by the partygenerating the metadata, although other deeper sub-categories, such asplayer names, field names, positional data, former player, etc. may beavailable in the nomenclature hierarchy for common use in otherapplications requiring such depth.

As an example of use of the deeper hierarchy, in the event that anadvertiser wishes to make use of the engine of the present invention,more specific hierarchy levels may be employed by an advertiserapplication using the metadata associated with the audio/visual work.For example, in the reference hereinabove to “purses”, Gucci may wish toadvertise its own brown purses in certain cases when the object purse isshown in an audio/visual work, but only a very specific purse. Thus,advertisers may make use of more specific hierarchy levels, such as,under the sub-category of “purses”, categories for “Gucci” purses, andthen “brown” Gucci purses (of course other purse colors, types of Guccipurses, model years of Gucci purses, etc., may be made of use at thislevel of the hierarchy), leading to the reference, by the metadata, toan advertisement external to the audio/visual work (which may beaccessible over one or more of the communication access points) only inthe event the viewer of the audio/visual work interacts with a brown,Gucci purse object. Needless to say in light of the disclosure herein,the nomenclature hierarchy of the present invention may include atranslator, whereby the nomenclature of one human language (or computerlanguage) is precisely and consistently translated into the commonterminology of another language, with no loss of commonality in anylanguage.

Therefore, although different levels of the nomenclature hierarchy maybe employed by different users of audio/visual works, or by differentapplications associated via the metadata with the audio/visual works,the hierarchal nomenclature references employed are the same at anyrespective level across all users, across all audio/video works, andacross all items, objects or video portions of that type. Of course,this aspect of the present invention makes available a number ofadvantageous presentations for association with audio/visual works. Forexample, in the exemplary embodiment discussed above, Gucci may wish toplace an overlay advertisement in the lower right hand corner of anyaudio/visual work making reference to purses, or may wish to place anoverlay advertisement only with respect to those audio/visual works thatmake reference to brown purse, or only brown, Gucci purses. However, itmust be noted that such choices are not made available in the prior artin any event in any audio/visual work, due to the fact that, without avideo engine having a common nomenclature hierarchy to create commonreferences across all audio/visual works, the lack of consistentreference to objects makes searching for multiple appearances of suchobjects or video portions across multiple audio/visual works difficultif not impossible.

In view of the video engine supplying a common hierarchical nomenclatureas discussed above, applications and/or audio/visual filters may bedeveloped to allow access to, interaction with, or reference toparticular items, objects, or video portions across all videos createdanywhere for play over any media. For example, in the exemplaryembodiment discussed above, a user may access a video filter or videoapplication that allows that user to record, or view, or buy, or thelike, by interaction with any reference to a brown Gucci purse in anyvideo across all videos. Such a filter or application may, of course,attempt to metadata tag only those audio/visual works deemed mostlikely, such as based on a prioritization filter, to make the requestedreference in the audio/visual work, or may crawl across all audio/visualworks on all media obtainable to the video engine via any mediaaccessible over the communication access points.

Of course, even using the video engine of the present invention tocreate a common nomenclature across consistent objects among all videoworks, the task of assessing a particular object or objects across agreat many video works may be overwhelming. Thus, the video engine ofthe present invention may be programmed with the aforementionedprioritization filter 68, whereby, based on a user type of the videoengine user, the prioritization filter 68 prioritizes the level of thehierarchy at which review is best to occur, the media type over whichreview of audio/visual works is best to occur, or the type ofcommunication access point that the most desired customers have thehighest likelihood of using, for example. Thus, the video engine of thepresent invention may make use of empirical data in the application ofthe nomenclature hierarchy to arrive at the most desired result ofnomenclature assignment for any particular application. Additionally,this empirical data may be accessed from any communication access point,and thus any media type, to which the video engine has access, such asby obtaining empirical video over the internet, from televisionbroadcasts, from the frequency of the play of certain commercials orother audio/visual works over internet, radio, personal electronicdevice, or television, and the like. Additionally, certain user typesmay be polled, such as by polling developers or advertisers as to themanner of prioritization for accessing audio/visual works havingparticular nomenclature therein. Thereby, the meta data in compliancewith the hierarchy may be exposed for use nakedly or in any availableapplication, or for development of applications or filters, by users,advertisers, and developers, etc.

Yet more specifically, it has been approximated that there are over onetrillion hours of digitized video available through a variety of mediasources. Each such audio/visual work may have corresponded theretometadata that is indicative of and allows for interaction with theaudio/video work, portions thereof and objects therein. Consequently, anadditional function performed by the video engine of the presentinvention may be prioritization of that video or those objects whichmust be primarily, secondarily, tertiarily, etc. associated withmetadata or a metadata stream. It will be apparent to those skilled inthe art that this prioritization may either form a portion of the videoengine, and may thereby mark different videos viewed by different userswith the same nomenclature hierarchy, while simultaneously reportingsuch marking of objects (and which videos or video portions have been somarked) back to a remote operations hub 70, or a remote operations hubmay likewise begin the process of marking videos with the nomenclaturehierarchy for feed to local video players alongside the audio/visualwork feed. Of course, in undertaking the prioritization of which videosshould be marked first (primarily), and which objects within whichvideos should be marked first, the focus may be on one or more of avariety of factors, including but not limited to: high desirability ofsponsorship for videos or objects; high volume of viewers of particularvideos, video types, or with desire to see particular objects; number oflikely references to particular objects and correspondent actionsnecessary by the nomenclature engine to name all such objects in allsuch videos; likely order of executed affiliations subscribing to thecommon nomenclature of the present invention. Other factors may, ofcourse, be apparent to those of ordinary skill in the art.

In one of the aforementioned exemplary embodiments, metadata within thecommon nomenclature may be prioritized for application to those objectshaving the highest desire for use by the highest desirabilityadvertisers. Thus, those objects that advertisers get premier return oninvestment for advertising in association with, and/or those items thatadvertisers otherwise most wished to be affiliated with, may provide anopportunity for the highest priority objects and or videos to be markedby the video engine in response to the prioritization instruction.Additionally and alternatively, metadata marking may be prioritized tothose audio/visual works that are most frequently watched by users, suchas broadcasts of the National Football League. By thus marking the mostpopular objects within the most popular programs, a variety of othereconomic avenues and applications may be opened, such as, using theexample of the NFL, advertisers placing highest priority on the highestwatched programs, and users having the greatest desire for interactivitywith the highest watched programs (such as through being fans of a teamor a participant in fantasy sports with regard to the NFL exampleabove). There is a likely increase in an advertiser's desire to beaffiliated with programming, and objects within such programming, thathave been the subject of such an indication that such programming and/orsuch objects are among the most watched or interacted with.

Prioritization thus may primarily target, for example, network shows inprime time. Further, such prime time shows may have a limited number ofobjects in each video frame, or may have objects that need be metadatatagged only once because they are re-used week after week. Such frequentobjects may include, for example, background sets that appear in manyscenes every week and that, when interacted with by a user, may have thesame metadata linked thereto every week, such as the New York Citytourism board and/or Wikipedia/New York if New York City is in thebackground often. Of course, as will be apparent to one skilled in theart in view of the discussion herein, in the aforementioned New Yorkexample, the video engine, through the prioritization, will insure thatthe reference to the New York skyline is consistent across all videos,which may thereby allow applications using the metadata associated viathat reference to make various manipulations based on the New Yorkskyline in any video, without having to view the video beforeprogramming the application. For example, and application may providethat, each time the New York skyline appears, in any video, the viewermay cursor over the skyline, click or hit enter, and be taken to the NewYork state tourism board. Similarly, due to the multiple levels of thenomenclature hierarchy, an application may link to a September 11memorial site if the user cursors over the pre-Sep. 11, 2001 New Yorkskyline in any video, but may link to a different location, via metadatatagging, if the user cursors over the New York skyline in any videoshowing the post 9/11 skyline. Of course, the interaction by the vieweritself may vary from application to application, and may include“mousing over” an object, clicking an object, calling up a menu oroverlay on a scene or an object, pausing the video to interact, notpausing the video, and the like.

Prioritization may additionally allow for differentiation of the worthof object, videos, or video portions, such as for advertisers. Forexample, a little known object in a home made online video may have lowpriority, and thus lower worth, such as for the purchase of anadvertisement to be associated with an object in such a video. However,a well known object in a prime time network television video may havehigh priority, and thus high worth, and may thereby demand premiumpayment for advertisements associated with such objects.

Steps may be taken to simplify the exercise undertaken by theprioritization, such as the reference above to having millions of videoengines marking, using the common nomenclature, millions of videos underthe supervision of a remote operations center. For example, in the eventthe common nomenclature is to be manually associated with objects in avideo work for sending outbound from the remote operations center to thelocal video players, the resolution of the audio/visual work used tomanually enter the corresponded metadata may be lower than thatdesirable for view by a user.

The prioritization of application of metadata to certain objects orvideos, or the depth of the hierarchy to which such applications areperformed, will vary in accordance with the user, the target, theapplication creator, and the like. For example, applications using themetadata built by or approved by professional sports leagues may makeuse of their own sports broadcasts in the applications they build atonly a very high level of the hierarchy, and only with respect to veryfew programming objects. As such, lower levels of the hierarchy, orother objects, may be more readily available in such league broadcaststo other application creators. However, applications for shows on afood-related channel may make use of very deep hierarchical levels, suchas food brands, kitchen utensils, expirations dates, and the like. Theintelligent prioritization of the present invention may elect, withregard to what videos and/or what objects, the vertical depth in thehierarchy for application of the common nomenclature metadata tagging ona case-by-case basis.

Although the invention has been described and pictured in an exemplaryform with a certain degree of particularity, it is understood that thepresent disclosure of the exemplary form has been made by way ofexample, and that numerous changes in the details of construction andcombination and arrangement of parts and steps may be made withoutdeparting from the spirit and scope of the invention as set forth in theclaims hereinafter.

The invention claimed is:
 1. A video engine for use with a video player,comprising: a presenter that presents at least one of a plurality ofaudio/visual works; a software application capable of a metadata-relatedinteraction with an identified object that appears within the presentedaudio/visual work; a communication access point over which metadataassociated with the presented audio/visual work is received, and overwhich at least a portion of the metadata-related interaction occurs; ahierarchical taxonomy that effects a common metadata reference to eachidentified appearance of the object in the plurality of works, whereinthe hierarchical taxonomy provides a consistent set of identifiers withwhich essentially any object that appears in any of the plurality ofaudio/video works can be identified with a degree of particularity; andan automated prioritization filter that selects from the audio/visualworks those deemed most likely to contain an appearance of the object.2. The video engine of claim 1, wherein the software application is atleast partially remote from the presenter.
 3. The video engine of claim1, wherein the metadata-related interaction comprises interaction withremotely generated metadata associated with the presented audio/visualwork.
 4. The video engine of claim 1, wherein the metadata-relatedinteraction comprises interaction with locally generated metadataassociated with the presented audio/visual work.
 5. The video engine ofclaim 1, wherein the communication access point provides access to oneof satellite communication, radio communication, wired communication,infrared communication, coaxial cable communication, WIFI communication,WiMAX communication, LAN communication, WAN communication, telephoniccommunication, DSL communication, and Ethernet communication.
 6. Thevideo engine of claim 1, wherein the hierarchical taxonomy comprises atleast three categories of breadth and at least three levels of depth forcommon metadata references.
 7. The video engine of claim 1, wherein atleast a portion of a plurality of common metadata references areautomatically associated with objects appearing in the plurality ofaudio/visual works.
 8. The video engine of claim 7, wherein theautomatic association occurs remotely from the presenter.
 9. The videoengine of claim 7, wherein the automatic association occurs locally tothe presenter.
 10. The video engine of claim 7, wherein the automaticassociation occurs via object recognition software.
 11. The video engineof claim 10, wherein the object recognition software is employed as partof a crawler that reviews the plurality of audio/visual works.
 12. Thevideo engine of claim 1, wherein at least a portion of a plurality ofcommon metadata references are manually associated with objectsappearing in the plurality of audio/visual works.
 13. The video engineof claim 12, wherein the manual association occurs remotely from thepresenter.
 14. The video engine of claim 12, wherein the manualassociation occurs locally to the presenter.
 15. The video engine ofclaim 1, wherein the prioritization filter prioritizes the order ofmaking common metadata references in each of the plurality of works. 16.The video engine of claim 1, wherein the prioritization filterprioritizes the level of the hierarchical taxonomy at which the commonmetadata references are reviewed for a particular one of a plurality ofmetadata-related interactions.
 17. The video engine of claim 1, whereinthe prioritization filter prioritizes the communication point over whichreview is to be made for particular ones of the common metadatareferences.
 18. The video engine of claim 1, wherein the prioritizationfilter makes use of empirical data external to audio/visual works toprioritize application of a plurality of common metadata references.